feat(fly-io,hetzner,railway): MachineIdentity, tagged createServer errors, slim GraphQL by sam-goodwin · Pull Request #477 · alchemy-run/distilled

sam-goodwin · 2026-08-26T20:44:39Z

Add Fly MachineIdentity so Machines PetSem encrypt/sign/decrypt/verify can talk to /.fly/api as the machine. Org-token GetSecret on the same fiber is unchanged.

Slim Railway GraphQL list/get selections so nuke list()s don't pull workspace/team/members/baseEnvironment/canvasGroupRefs/subscriptionType — those fields fail decode on live rows and blow the rate budget. Stop paginateRelay when endCursor does not advance. Type GraphQL INTERNAL_SERVER_ERROR "not found" / "creating too quickly" as RailwayNotFound / RailwayRateLimited, and parse retry-after from "per N seconds" or "every N s" (default 31s when the gateway sends none). Identify GraphQL as source=alchemy so Cloudflare's anonymous 10 RPS cap does not apply.

Type Hetzner createServer quota/placement failures as ServerLimitExceeded / ServerPlacementError instead of Forbidden / PreconditionFailed plus message text. convert.ts applies /shapes/ patches after OpenAPI conversion.

Fly MachineIdentity

FlyIoProtocol still sends Authorization: Bearer by default. A per-call MachineIdentity service drops that header and sets Connection: close. The protocol does not sniff an empty API key.

export class MachineIdentity extends Context.Service<
  MachineIdentity,
  true
>()("FlyIoMachineIdentity") {}

Generated machines ops are unchanged. Callers provide MachineIdentity plus an HttpClient with socketPath: "/.fly/api".

Hetzner createServer errors

// 403 resource_limit_exceeded, message includes "server limit"
ServerLimitExceeded
// 412 resource_unavailable, message includes "placement"
ServerPlacementError

Railway GraphQL

# projects list — identity scalars
project { id name workspaceId baseEnvironmentId primaryEnvironmentId }

# project({id}) — identity plus services / buckets / groups
services { id name groupId templateId }
buckets  { id name groupId }
groups   { id name groupId }

# environments list — identity plus volumeInstances (no environment({id}))
# volumeInstance GET — no nested environment
# volumeCreate — returns volumeInstances

# Boolean! mutations — no subfields
mutation deploymentCancel($id: String!) { deploymentCancel(id: $id) }

environment({id}) still selects canvasGroupRefs / meta. List paths must not call it.

const retryAfterForRateLimit = (message, headers) => {
  const fromHeader = parseRetryAfterForStatus(429, headers);
  if (fromHeader !== undefined) return fromHeader;
  const match =
    /per (\d+) seconds/.exec(message) ?? /every (\d+)\s*s/.exec(message);
  if (match !== null) return Duration.seconds(Number(match[1]) + 1);
  return Duration.seconds(31);
};

FlyIoProtocol still sends Bearer by default. When MachineIdentity is provided on the calling fiber, encode drops Authorization and sets Connection: close so encrypt/sign/decrypt/verify work as the machine over the unix socket without sniffing an empty token.

alchemy-version-bot · 2026-08-26T20:46:46Z

Install the packages built from this commit:

Distilled

@distilled.cloud/core

bun add https://pkg.ing/@distilled.cloud/core/88b462a

@distilled.cloud/aws

bun add https://pkg.ing/@distilled.cloud/aws/88b462a

@distilled.cloud/axiom

bun add https://pkg.ing/@distilled.cloud/axiom/88b462a

@distilled.cloud/azure

bun add https://pkg.ing/@distilled.cloud/azure/88b462a

@distilled.cloud/cloudflare

bun add https://pkg.ing/@distilled.cloud/cloudflare/88b462a

@distilled.cloud/coinbase

bun add https://pkg.ing/@distilled.cloud/coinbase/88b462a

@distilled.cloud/discord

bun add https://pkg.ing/@distilled.cloud/discord/88b462a

@distilled.cloud/expo-eas

bun add https://pkg.ing/@distilled.cloud/expo-eas/88b462a

@distilled.cloud/fly-io

bun add https://pkg.ing/@distilled.cloud/fly-io/88b462a

@distilled.cloud/gcp

bun add https://pkg.ing/@distilled.cloud/gcp/88b462a

@distilled.cloud/github

bun add https://pkg.ing/@distilled.cloud/github/88b462a

@distilled.cloud/hetzner

bun add https://pkg.ing/@distilled.cloud/hetzner/88b462a

@distilled.cloud/kubernetes

bun add https://pkg.ing/@distilled.cloud/kubernetes/88b462a

@distilled.cloud/mongodb-atlas

bun add https://pkg.ing/@distilled.cloud/mongodb-atlas/88b462a

@distilled.cloud/neon

bun add https://pkg.ing/@distilled.cloud/neon/88b462a

@distilled.cloud/planetscale

bun add https://pkg.ing/@distilled.cloud/planetscale/88b462a

@distilled.cloud/posthog

bun add https://pkg.ing/@distilled.cloud/posthog/88b462a

@distilled.cloud/prisma-postgres

bun add https://pkg.ing/@distilled.cloud/prisma-postgres/88b462a

@distilled.cloud/railway

bun add https://pkg.ing/@distilled.cloud/railway/88b462a

@distilled.cloud/stripe

bun add https://pkg.ing/@distilled.cloud/stripe/88b462a

@distilled.cloud/supabase

bun add https://pkg.ing/@distilled.cloud/supabase/88b462a

@distilled.cloud/turso

bun add https://pkg.ing/@distilled.cloud/turso/88b462a

@distilled.cloud/typesense

bun add https://pkg.ing/@distilled.cloud/typesense/88b462a

@distilled.cloud/vercel

bun add https://pkg.ing/@distilled.cloud/vercel/88b462a

@distilled.cloud/workos

bun add https://pkg.ing/@distilled.cloud/workos/88b462a

The GraphQL `projects` connection was selecting nested workspace/team/members/baseEnvironment at maxDepth 2. That pulled subscriptionType and canvasGroupRefs, failed decode on live rows, and made listOwnedProjects hang under nuke. Also terminate paginateRelay when endCursor does not advance.

Nuke scans ~20 resource list()s that each call project({id}) and environments. The default maxDepth-2 selection pulled workspace/team/ members/baseEnvironment and canvasGroupRefs, then distilled retried 429s silently for minutes. project({id}) now selects identity plus services/buckets/groups. environments list is identity scalars. HttpClient GraphQlHttpGate caps in-flight GraphQL POSTs at 2 (DISTILLED_RAILWAY_GRAPHQL_CONCURRENCY).

Do not serialize GraphQL behind a process lock. Slim project({id}) and environments selections are the nuke-scan fix; wrapping HttpClient broke .items() streams.

Volume.list can enumerate from the slim environments connection. Nuke must not call environment({id}) — that GET still selects canvasGroupRefs/meta and is what made the scan spin.

Select volumeInstances on volumeCreate and environments list, drop the nested environment from volumeInstance GET, stamp service.templateId on the slim project query, and map ServiceInstance/BucketInstance-not-found plus "creating too quickly" onto RailwayNotFound/RailwayRateLimited. GraphQL rate limits without Retry-After default to 31s.

S3 (and Tigris) report a missing object as NoSuchKey. deleteObject was patched to NotFound, so Fly.Website.AssetDeployment could not catchTag the same error AWS S3 getObject uses.

…nCreateFailed GraphQL INTERNAL_SERVER_ERROR "Failed to create service domain" is a transient instance-not-ready. Also map environment-create rate limits.

…loyment mutations Hetzner createServer now surfaces ServerLimitExceeded and ServerPlacementError instead of Forbidden/PreconditionFailed message checks. convert.ts applies /shapes/ patches after OpenAPI conversion. Railway deploymentCancel/Remove/Stop/Restart queries match Boolean! (no subfields). GraphQL requests send source=alchemy so the public edge does not apply the anonymous 10 RPS cap, and retry-after also parses "every N s".

sam-goodwin added 5 commits August 26, 2026 17:08

revert(railway): drop GraphQlHttpGate

48435b0

Do not serialize GraphQL behind a process lock. Slim project({id}) and environments selections are the nuke-scan fix; wrapping HttpClient broke .items() streams.

fix(railway): select volumeInstances on environments list

9f24dc6

Volume.list can enumerate from the slim environments connection. Nuke must not call environment({id}) — that GET still selects canvasGroupRefs/meta and is what made the scan spin.

sam-goodwin changed the title ~~feat(fly-io): MachineIdentity protocol for PetSem over /.fly/api~~ fix: fly and railway patches Aug 28, 2026

sam-goodwin changed the title ~~fix: fly and railway patches~~ feat(fly-io,railway): MachineIdentity protocol and slim GraphQL queries Aug 28, 2026

sam-goodwin added 3 commits August 28, 2026 10:14

fix(aws/s3): type deleteObject missing-key as NoSuchKey

e2ebe96

S3 (and Tigris) report a missing object as NoSuchKey. deleteObject was patched to NotFound, so Fly.Website.AssetDeployment could not catchTag the same error AWS S3 getObject uses.

fix(railway): type service-domain create retry as RailwayServiceDomai…

85972de

…nCreateFailed GraphQL INTERNAL_SERVER_ERROR "Failed to create service domain" is a transient instance-not-ready. Also map environment-create rate limits.

sam-goodwin changed the title ~~feat(fly-io,railway): MachineIdentity protocol and slim GraphQL queries~~ feat(fly-io,hetzner,railway): MachineIdentity, tagged createServer errors, slim GraphQL Aug 28, 2026

sam-goodwin merged commit e7f12dd into main Aug 29, 2026
3 checks passed

sam-goodwin deleted the feat/fly-machine-identity branch August 29, 2026 02:56

Navigation Menu

Uh oh!

feat(fly-io,hetzner,railway): MachineIdentity, tagged createServer errors, slim GraphQL - #477

feat(fly-io,hetzner,railway): MachineIdentity, tagged createServer errors, slim GraphQL#477
sam-goodwin merged 9 commits into
mainfrom
feat/fly-machine-identity

sam-goodwin commented Aug 26, 2026 •

edited

Loading

Uh oh!

alchemy-version-bot Bot commented Aug 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

sam-goodwin commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fly MachineIdentity

Hetzner createServer errors

Railway GraphQL

Uh oh!

alchemy-version-bot Bot commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Distilled

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sam-goodwin commented Aug 26, 2026 •

edited

Loading

alchemy-version-bot Bot commented Aug 26, 2026 •

edited

Loading